3D Transition Matrix Solution for a Path Dependency Problem of Markov Chains-Based Prediction in Cellular Networks

Handover (HO) management is one of the critical challenges in current and future mobile communication systems due to new technologies being deployed at a network level, such as small and femtocells. Because of the smaller sizes of cells, users are expected to perform more frequent HOs, which can increase signaling costs and also decrease user's performance, if a HO is performed poorly. In order to address this issue, predictive HO techniques, such as Markov chains (MC), have been introduced in the literature due to their simplicity and generality. This technique, however, experiences a path dependency problem, specially when a user performs a HO to the same cell, also known as a re-visit. In this paper, the path dependency problem of this kind of predictors is tackled by introducing a new 3D transition matrix, which has an additional dimension representing the orders of HOs, instead of a conventional 2D one. Results show that the proposed algorithm outperforms the classical MC based predictors both in terms of accuracy and HO cost when re-visits are considered

Introducing a Novel Minimum Accuracy Concept for Predictive Mobility Management Schemes

In this paper, an analytical model for the minimum required accuracy for predictive methods is derived in terms of both handover (HO) delay and HO signaling cost. After that, the total HO delay and signaling costs are derived for the worst-case scenario (when the predictive process has the same performance as the conventional one), and simulations are conducted using a cellular environment to reveal the importance of the proposed minimum accuracy framework. In addition to this, three different predictors; Markov Chains, Artificial Neural Network (ANN) and an Improved ANN (IANN) are implemented and compared. The results indicate that under certain circumstances, the predictors can occasionally fall below the applicable level. Therefore, the proposed concept of minimum accuracy plays a vital role in determining this corresponding threshold

Self-organization for 5G and beyond mobile networks using reinforcement learning

The next generations of mobile networks 5G and beyond, must overcome current networks limitations as well as improve network performance. Some of the requirements envisioned for future mobile networks are: addressing the massive growth required in coverage, capacity and traffic; providing better quality of service and experience to end users; supporting ultra high data rates and reliability; ensuring latency as low as one millisecond, among others. Thus, in order for future networks to enable all of these stringent requirements, a promising concept has emerged, self organising networks (SONs). SONs consist of making mobile networks more adaptive and autonomous and are divided in three main branches, depending on their use-cases, namely: self-configuration, self-optimisation, and self-healing. SON is a very promising and broad concept, and in order to enable it, more intelligence needs to be embedded in the mobile network. As such, one possible solution is the utilisation of machine learning (ML) algorithms. ML has many branches, such as supervised, unsupervised and Reinforcement Learning (RL), and all can be used in different SON use-cases. The objectives of this thesis are to explore different RL techniques in the context of SONs, more specifically in self-optimization use-cases. First, the use-case of user-cell association in future heterogeneous networks is analysed and optimised. This scenario considers not only Radio Access Network (RAN) constraints, but also in terms of the backhaul. Based on this, a distributed solution utilizing RL is proposed and compared with other state-of-the-art methods. Results show that the proposed RL algorithm outperforms current ones and is able to achieve better user satisfaction, while minimizing the number of users in outage. Another objective of this thesis is the evaluation of Unmanned Aerial vehicles (UAVs) to optimize cellular networks. It is envisioned that UAVs can be utilized in different SON use-cases and integrated with RL algorithms to determine their optimal 3D positions in space according to network constraints. As such, two different mobile network scenarios are analysed, one emergency and a pop-up network. The emergency scenario considers that a major natural disaster destroyed most of the ground network infrastructure and the goal is to provide coverage to the highest number of users possible using UAVs as access points. The second scenario simulates an event happening in a city and, because of the ground network congestion, network capacity needs to be enhanced by the deployment of aerial base stations. For both scenarios different types of RL algorithms are considered and their complexity and convergence are analysed. In both cases it is shown that UAVs coupled with RL are capable of solving network issues in an efficient and quick manner. Thus, due to its ability to learn from interaction with an environment and from previous experience, without knowing the dynamics of the environment, or relying on previously collected data, RL is considered as a promising solution to enable SON

A survey of machine learning techniques applied to self organizing cellular networks

In this paper, a survey of the literature of the past fifteen years involving Machine Learning (ML) algorithms applied to self organizing cellular networks is performed. In order for future networks to overcome the current limitations and address the issues of current cellular systems, it is clear that more intelligence needs to be deployed, so that a fully autonomous and flexible network can be enabled. This paper focuses on the learning perspective of Self Organizing Networks (SON) solutions and provides, not only an overview of the most common ML techniques encountered in cellular networks, but also manages to classify each paper in terms of its learning solution, while also giving some examples. The authors also classify each paper in terms of its self-organizing use-case and discuss how each proposed solution performed. In addition, a comparison between the most commonly found ML algorithms in terms of certain SON metrics is performed and general guidelines on when to choose each ML algorithm for each SON function are proposed. Lastly, this work also provides future research directions and new paradigms that the use of more robust and intelligent algorithms, together with data gathered by operators, can bring to the cellular networks domain and fully enable the concept of SON in the near future

Blockchain-enabled resource management and sharing for 6G communications

The sixth-generation (6G) network must provide performance superior to previous generations to meet the requirements of emerging services and applications, such as multi-gigabit transmission rate, even higher reliability, and sub 1 ms latency and ubiquitous connection for the Internet of Everything (IoE). However, with the scarcity of spectrum resources, efficient resource management and sharing are crucial to achieving all these ambitious requirements. One possible technology to achieve all this is the blockchain. Because of its inherent properties, the blockchain has recently gained an important position, which is of great significance to 6G network and other networks. In particular, the integration of the blockchain in 6G will enable the network to monitor and manage resource utilization and sharing efficiently. Hence, in this paper, we discuss the potentials of the blockchain for resource management and sharing in 6G using multiple application scenarios, namely, Internet of things, device-to-device communications, network slicing, and inter-domain blockchain ecosystems

Distributed Learning Based Handoff Mechanism for Radio Access Network Slicing with Data Sharing

Network slicing (NS) has been identified as a fundamental technology for future mobile networks to meet extremely diverse communication requirements by providing tailored quality of service (QoS). However, due to the introduction of NS into radio access networks (RAN) forming a UE-BS-NS three-layer association, handoff becomes very complicated and cannot be resolved by conventional policies. In this paper, we propose a multi-agent reinforcement LEarning based Smart handoff policy with data Sharing, named LESS, to reduce handoff cost while maintaining user QoS requirements in RAN slicing. Considering the large action space introduced by multiple users and the data sparsity problem due to user mobility, LESS is designed to have two components: 1) LESS-DL, a modified distributed Q-learning algorithm with small action space to make handoff decisions; 2) LESS-DS, a data sharing mechanism using limited data to improve the accuracy of handoff decisions made by LESS-DL. The proposed LESS mechanism uses LESS-DL to choose both the target base station and NS when a handoff occurs, and then updates the Q-values of each user according to LESS-DS. Numerical results show that in typical scenarios, LESS can significantly reduce the handoff cost when compared with traditional handoff policies without learning

Efficient handover mechanism for radio access network slicing by exploiting distributed learning

Network slicing is identified as a fundamental architectural technology for future mobile networks since it can logically separate networks into multiple slices and provide tailored quality of service (QoS). However, the introduction of network slicing into radio access networks (RAN) can greatly increase user handover complexity in cellular networks. Specifically, both physical resource constraints on base stations (BSs) and logical connection constraints on network slices (NSs) should be considered when making a handover decision. Moreover, various service types call for an intelligent handover scheme to guarantee the diversified QoS requirements. As such, in this paper, a multi-agent reinforcement LEarning based Smart handover Scheme, named LESS, is proposed, with the purpose of minimizing handover cost while maintaining user QoS. Due to the large action space introduced by multiple users and the data sparsity caused by user mobility, conventional reinforcement learning algorithms cannot be applied directly. To solve these difficulties, LESS exploits the unique characteristics of slicing in designing two algorithms: 1) LESS-DL, a distributed Q-learning algorithm to make handover decisions with reduced action space but without compromising handover performance; 2) LESS-QVU, a modified Q-value update algorithm which exploits slice traffic similarity to improve the accuracy of Q-value evaluation with limited data. Thus, LESS uses LESS-DL to choose the target BS and NS when a handover occurs, while Q-values are updated by using LESS-QVU. The convergence of LESS is theoretically proved in this paper. Simulation results show that LESS can significantly improve network performance. In more detail, the number of handovers, handover cost and outage probability are reduced by around 50%, 65%, and 45%, respectively, when compared with traditional methods

IMPRESS: indoor mobility prediction framework for pre-emptive indoor-outdoor handover for mmwave networks

Millimeter-wave (mmWave) communication, the main success behind the fifth generation of mobile communication networks, will increase the ultra-dense small cell deployment under its limited coverage characteristics. Therefore, providing a seamless connection to its users, to whom transitioning between indoor and outdoor in a heterogeneous network environment particularly is a significant issue that needs to be addressed. In this paper, we present a two-fold contribution with a comprehensive study on mm-wave handovers. A user-based indoor mobility prediction via Markov chain with an initial transition matrix is proposed in the first step. Based on this acquired knowledge of the user’s movement pattern in the indoor environment, we present a pre-emptive handover algorithm in the second step. This algorithm aims to keep the QoS high for indoor users when transitioning between indoor and outdoor in a heterogeneous network environment. The proposed algorithm shows a reduction in the handover signalling cost by more than 50%, outperforming conventional handover algorithms